Future Person Localization in First-Person Videos

We present a new task that predicts future locations of people observed in first-person videos. Consider a first-person video stream continuously recorded by a wearable camera. Given a short clip of a person that is extracted from the complete stream, we aim to predict that person's location in future frames. To facilitate this future person localization ability, we make the following three key observations: a) First-person videos typically involve significant ego-motion which greatly affects the location of the target person in future frames; b) Scales of the target person act as a salient cue to estimate a perspective effect in first-person videos; c) First-person videos often capture people up-close, making it easier to leverage target poses (e.g., where they look) for predicting their future locations. We incorporate these three observations into a prediction framework with a multi-stream convolution-deconvolution architecture. Experimental results reveal our method to be effective on our new dataset as well as on a public social interaction dataset.Comment: Accepted to CVPR 201

一人称視点映像における人物位置予測

学位の種別: 修士University of Tokyo(東京大学

Object Instance Identification in Dynamic Environments

We study the problem of identifying object instances in a dynamic environment where people interact with the objects. In such an environment, objects' appearance changes dynamically by interaction with other entities, occlusion by hands, background change, etc. This leads to a larger intra-instance variation of appearance than in static environments. To discover the challenges in this setting, we newly built a benchmark of more than 1,500 instances built on the EPIC-KITCHENS dataset which includes natural activities and conducted an extensive analysis of it. Experimental results suggest that (i) robustness against instance-specific appearance change (ii) integration of low-level (e.g., color, texture) and high-level (e.g., object category) features (iii) foreground feature selection on overlapping objects are required for further improvement.Comment: Joint 1st Ego4D and 10th EPIC Workshop (EPIC@CVPR2022) Extended Abstrac

Recurrent femoral shaft fractures in a child with gnathodiaphyseal dysplasia: a case report

Abstract Background Gnathodiaphyseal dysplasia (GDD) is an extremely rare autosomal dominant disease characterized by cemento-osseous lesions in the jawbones, bone fragility, and diaphyseal sclerosis of the tubular bones. Patients with GDD are prone to sustain fractures by minor accidents. Although over 80 cases have been reported, detailed information about the orthopedic treatment of the fractures is limited. Case presentation A 9-year-old Japanese girl with a known history of GDD presented with pain and deformity in the left thigh after a minor fall. She had a displaced transverse fracture in the mid-shaft of the left femur and underwent a closed reduction and external fixation. In the 25th week after the initial surgery, she had another fracture in the left femur at one of the half-pin insertion sites. She underwent an external fixation again. After this operation, the patient sustained another refracture at the same fracture site and one supracondylar fracture at the distant site of the femur. The supracondylar fracture occurred without any triggering activity before beginning a weight-bearing exercise. The supracondylar fracture was successfully treated conservatively, but she sustained two more diaphyseal fractures at half-pin insertion sites one after another. She eventually underwent a revision surgery with a flexible intramedullary nail. At 3 months postoperatively, the fracture was healed and the patient maintained her ambulatory status without further refracture. Conclusions Patients with GDD might have narrower safety ranges of biomechanical and physiological drawbacks, which are considered to be acceptable in ordinary cases. The choice of treatment should be aimed at minimizing these negative effects. We recommend intramedullary devise as the first-choice implant for the treatment of isolated femoral shaft fracture in GDD patients in this age group